On the Communication Complexity of Distributed Set-Joins

Given a set-comparison predicateP and given two lists of setsA = (A1, . . . , Am) and B = (B1, . . . , Bm), with all Ai, Bj ⊆ [n], the P-set join A ./P B is defined to be the set {(i, j) ∈ [n] × [n] | P(Ai, Bj)} ([n] denotes {1, 2, . . . , n}). When P(Ai, Bj) is the condition “Ai ∩ Bj 6= ∅” we call this the set-intersection-notempty join (a.k.a. the composition ofA and B); whenP(Ai, Bj) is “Ai∩Bj = ∅” we call it the set-disjointness join; when P(Ai, Bj) is “Ai = Bj” we call it the set-equality join; when P(Ai, Bj) is “|Ai ∩ Bj | ≥ T ” for a given threshold T , we call it the setintersection threshold join. Assuming A and B are stored at two different sites in a distributed environment, we study the (randomized) communication complexity of computing these, and related, set-joins A ./P B, as well as the (randomized) communication complexity of computing the exact and approximate value of their size k = |A ./P B|. Combined, our analyses shed new insights into the quantitative differences between these different set-joins. Furthermore, given the close affinity of the natural join and the setintersection-not-empty join, our results also yield communication complexity results for computing the natural join in a distributed